Head mounted display having a plurality of display modes

ABSTRACT

Methods and apparatus provide for controlling an information processing apparatus that is connected to an image display apparatus worn on a head of a user, including: producing a planar image representing virtual objects in a virtual world, wherein the planar image includes characteristics that require the image display apparatus to be a non-see-through image display apparatus that does not allow the user to visually recognize real world objects outside the virtual world; and displaying the planar image on the image display apparatus in a display mode that changes a shape of the planar image as a function of a change in position and/or orientation of the image display apparatus, wherein the change in the shape of the planar image includes presenting the planar image in an orientation other than a front direction of the user.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent Application No.16,782,212, accorded a filing date of Feb. 5, 2020 (allowed), which is acontinuation of U.S. patent application Ser. No. 15/754,362, accorded afiling date of Feb. 22, 2018, issued as U.S. Pat. No. 10,594,936 on Mar.17, 2020, which is a National Stage application claiming priority toInternational Application No. PCT/JP2016/063862, filed on May 10, 2016,which claims priority to JP 2015-188025, filed Sep. 25, 2015, the entiredisclosures of which are hereby incorporated by reference.

TECHNICAL FIELD

The present invention relates to an information processing apparatus, acontrol method of a video display apparatus, and a program that displaya video on a video display apparatus used by being worn on the head by auser.

BACKGROUND ART

Some video display apparatuses such as head-mounted displays are used bybeing worn on the head by users. A video display apparatus of this kindallows a user to view an image by forming the image in front of theuser's eyes.

Some among the above video display apparatuses detect the user's headmotion and display an image that is updated with change in position ororientation of the user's head. Such a video display apparatus allowsviewing of videos that leave a user a “you are there” feeling, therebyfinding application in virtual reality technology and so on. As anexample, there is known a program that, while changing the orientationof a virtual camera placed in a virtual space to match with the user'shead motion, displays a video indicating how the virtual space looks asseen from the virtual camera on a video display apparatus. This allowsthe user to view a video that leaves the user a sensation as if he orshe were in a virtual space.

SUMMARY Technical Problem

In the above video display apparatus, due consideration has not beengiven enough to how planar videos should be displayed (the sametwo-dimensional videos as displayed by conventional displayapparatuses).

The present invention has been made in consideration of the abovecircumstances, and it is an object of the present invention to providean information processing apparatus, a control method of a video displayapparatus, and a program that can display a planar video in a mannersuitable for circumstances of use when a planar video is displayed onthe video display apparatus used by being worn on the head by a user.

Solution to Problem

An information processing apparatus according to the present inventionis an information processing apparatus connected to a video displayapparatus used by being worn on the head by a user. The informationprocessing apparatus includes a first display control section, a seconddisplay control section, and a switching control section. The firstdisplay control section displays a planar video on the video displayapparatus in a first display mode in which the planar video is displayedin the front direction of the user irrespective of the change inposition and/or orientation of the video display apparatus. The seconddisplay control section displays a planar video on the video displayapparatus in a second display mode in which the planar video isdisplayed in a manner that changes with the change in position and/ororientation of the video display apparatus. The switching controlsection switches between the first display mode and the second displaymode based on a given condition.

Also, a control method of a video display apparatus according to thepresent invention is a control method of a video display apparatus usedby being worn on the head by a user. The video display apparatus controlmethod includes a step of displaying a planar video on the video displayapparatus in a first display mode in which the planar video is displayedin the front direction of the user irrespective of the change inposition and/or orientation of the video display apparatus, a step ofdisplaying a planar video on the video display apparatus in a seconddisplay mode in which the planar video is displayed in a manner thatchanges with the change in position and/or orientation of the videodisplay apparatus, and a step of switching between the first displaymode and the second display mode based on a given condition.

Also, a program according to the present invention is a program causinga computer connected to a video display apparatus used by being worn onthe head by a user to function as first display control means, seconddisplay control means, and switching control means. The first displaycontrol means displays a planar video on the video display apparatus ina first display mode in which the planar video is displayed in the frontdirection of the user irrespective of the change in position and/ororientation of the video display apparatus. The second display controlmeans displays the planar video on the video display apparatus in asecond display mode in which the planar video is displayed in a mannerthat changes with the change in position and/or orientation of the videodisplay apparatus. The switching control means switches between thefirst display mode and the second display mode based on a givencondition. This program may be provided stored in a computer-readableand non-temporary information storage medium.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a configuration block diagram illustrating a configuration ofa video display system including an information processing apparatusaccording to an embodiment of the present invention.

FIG. 2 is a front perspective view illustrating an example of appearanceof a video display apparatus.

FIG. 3 is a rear perspective view illustrating an example of appearanceof the video display apparatus.

FIG. 4 is a functional block diagram illustrating functions of theinformation processing apparatus according to the present embodiment.

FIG. 5 is a diagram describing a specific example of a drawing processin a second display mode.

FIG. 6A is a diagram illustrating a display example in an initial stateof first and second display modes.

FIG. 6B is a diagram illustrating a display example of the first displaymode.

FIG. 6C is a diagram illustrating a display example of the seconddisplay mode.

FIG. 7 is a flowchart illustrating an example of a processing flowcarried out by the information processing apparatus according to thepresent embodiment.

DESCRIPTION OF EMBODIMENT

A detailed description will be given below of an embodiment of thepresent invention with reference to drawings.

FIG. 1 is a configuration block diagram illustrating a configuration ofa video display system 1 including an information processing apparatus10 according to an embodiment of the present invention. As illustratedin FIG. 1 , the video display system 1 is configured to include theinformation processing apparatus 10, an operating device 20, a camera25, a relay apparatus 30, and a video display apparatus 40.

The information processing apparatus 10 is an apparatus that supplies avideo to be displayed by the video display apparatus 40 and may be, forexample, a home gaming machine, a portable gaming machine, a personalcomputer, a smartphone, or a tablet. As illustrated in FIG. 1 , theinformation processing apparatus 10 is configured to include a controlsection 11, a storage section 12, and an interface section 13.

The control section 11 includes at least a processor such as CPU(Central Processing Unit) and performs a variety of informationprocessing tasks by executing a program stored in the storage section12. It should be noted that a specific example of a process performed bythe control section 11 in the present embodiment will be describedlater. The storage section 12 includes at least a memory device such asRAM (Random Access Memory) and stores a program executed by the controlsection 11 and data processed by the program.

The interface section 13 is an interface for data communication betweenthe operating device 20, the camera 25, and the relay apparatus 30. Theinformation processing apparatus 10 is connected to each of devicesincluding the operating device 20, the camera 25, and the relayapparatus 30 via the interface section 13 in a wired or wirelessfashion. Specifically, the interface section 13 may include a multimediainterface such as HDMI (High-Definition Multimedia Interface: registeredtrademark) to send video and audio, supplied by the informationprocessing apparatus 10, to the relay apparatus 30. The interfacesection 13 also includes a data communication interface such asBluetooth (registered trademark) or USB (Universal Serial Bus). Theinformation processing apparatus 10 receives various information fromthe video display apparatus 40 by way of the relay apparatus 30 andsends control signals and so on via this data communication interface.The information processing apparatus 10 receives a signal indicatingdetails of user operation input to the operating device 20 and imagedata captured by the camera 25 via this data communication interface.

The operating device 20 is a controller of home gaming machine or thelike for the user to perform various operations and issue instructionsto the information processing apparatus 10. Details of user operationinput to the operating device 20 are sent to the information processingapparatus 10 in a wired or wireless fashion. It should be noted that theoperating device 20 may include operating buttons, touch panel, and soon placed on the front face of the housing of the information processingapparatus 10. The camera 25 is installed on the front of the user usingthe present video display system 1 and captures a video including thevideo display apparatus 40 worn by the user.

The relay apparatus 30 is connected to the video display apparatus 40 ina wired or wireless fashion, accepts video data supplied from theinformation processing apparatus 10, and outputs the video signal thatmatches with the accepted data to the video display apparatus 40. Atthis time, the relay apparatus 30 may perform, as occasion demands, acorrection process that cancels out distortion that occurs in optics ofthe video display apparatus 40 for the supplied video data and output acorrected video signal. It should be noted that the video signalsupplied from the relay apparatus 30 to the video display apparatus 40includes two videos, one for left eye and another for right eye. Therelay apparatus 30 also relays various information exchanged between theinformation processing apparatus 10 and the video display apparatus 40such as audio data and control signals in addition to video data.

The video display apparatus 40 displays a video that matches with thevideo signal input from the relay apparatus 30 and allows the user toview the video. The video display apparatus 40 is a video displayapparatus used by being worn on the head by the user and supports videoviewing with both eyes. That is, the video display apparatus 40 forms avideo in front of each of the right and left eyes of the user. The videodisplay apparatus 40 is also configured to be able to display astereoscopic video using binocular parallax. As illustrated in FIG. 1 ,the video display apparatus 40 is configured to include a video displaydevice 41, an optical device 42, a light-emitting device 43, a motionsensor 44, a proximity sensor 45, and a communication interface 46.Further, FIGS. 2 and 3 illustrate appearance of the video displayapparatus 40, and FIG. 2 is a perspective view as seen from the front,and FIG. 3 is a perspective view as seen from the rear.

The video display device 41 is, for example, an organic EL(Electroluminescence) display panel or liquid crystal display panel anddisplays a video that matches with the video signal supplied from therelay apparatus 30. The video display device 41 displays two videos, onefor left eye and another for right eye. It should be noted that thevideo display device 41 may be a single display device that displaysvideos for left and right eyes side by side or may include two displaydevices each of which independently displays a video. Alternatively, aknown smartphone or other device may be used as the video display device41.

The optical device 42 is, for example, a hologram, a prism, or ahalf-mirror and placed in front of the user's eyes to transmit orrefract light of the videos displayed by the video display device 41 sothat the light enters each of the left and right eyes of the user.Specifically, a video for left eye displayed by the video display device41 enters the user's left eye by way of the optical device 42, and avideo for right eye enters the user's right eye by way of the opticaldevice 42. This allows the user to view the video for left eye with hisor her left eye and the video for right eye with his or her right eyewhile wearing the video display apparatus 40 on the head. It should benoted that, in the present embodiment, we assume that the video displayapparatus 40 is a non-see-through video display apparatus that does notallow the user to visually recognize the outside scene.

The light-emitting device 43 is, for example, an LED (Light EmittingDiode), is placed on the front of the video display apparatus 40, andemits light in response to an instruction from the informationprocessing apparatus 10. The information processing apparatus 10 canidentify the position of the video display apparatus 40 in the field ofview of the camera 25 by imaging light of the light-emitting device 43with the camera 25. That is, the information processing apparatus 10 cantrack the change in position and tilt of the video display apparatus 40using light of the light-emitting device 43. It should be noted that, asillustrated in FIG. 2 , the plurality of light-emitting devices 43 areprovided in the present embodiment.

The motion sensor 44 measures various information about the position,orientation, and motion of the video display apparatus 40. For example,the motion sensor 44 may include an acceleration sensor, a gyroscope, ora geomagnetic sensor. Measurement results of the motion sensor 44 aresent to the information processing apparatus 10 by way of the relayapparatus 30. The information processing apparatus 10 can usemeasurement results of the motion sensor 44 to identify the change inmotion and orientation of the video display apparatus 40. Specifically,the information processing apparatus 10 can detect the tilt andtranslation of the video display apparatus 40 relative to the verticaldirection by using measurement results of the acceleration sensor. Theinformation processing apparatus 10 can also detect the rotationalmotion of the video display apparatus 40 by using measurement results ofa gyroscope or a geomagnetic sensor.

The proximity sensor 45 is a sensor that detects, in a noncontactfashion, the presence of an object when the object approaches a targetdetection range near the sensor. The proximity sensor 45 may be of anytype such as infrared sensor, for example. As illustrated in FIG. 3 ,the proximity sensor 45 is placed approximately at the center on therear face of a main body portion of the video display apparatus 40. Thisallows the proximity sensor 45 to detect the approaching forehead of theuser when the user attempts to put on the video display apparatus 40 onthe head. The information processing apparatus 10 can detect the user'sputting on the video display apparatus 40 or taking it off from the headby using measurement results of the proximity sensor 45.

The communication interface 46 is an interface for carrying out datacommunication with the relay apparatus 30. For example, when the videodisplay apparatus 40 sends or receives data to or from the relayapparatus 30 by wireless communication such as wireless LAN (Local AreaNetwork) or Bluetooth, the communication interface 46 includes acommunication antenna and a communication module. The communicationinterface 46 may also include a communication interface such as HDMI orUSB to carry out data communication with the relay apparatus 30 in awired manner.

A description will be given next of the functions realized by theinformation processing apparatus 10 using FIG. 4 . As illustrated inFIG. 4 , the information processing apparatus 10 functionally includes avideo supply section 51, a first display control section 52, a seconddisplay control section 53, and a switching control section 54. Thesefunctions are realized as a result of execution of a program stored inthe storage section 12 by the control section 11. This program may beprovided to the information processing apparatus 10 via a communicationnetwork such as the Internet or may be provided stored in acomputer-readable information storage medium such as optical disc.

The video supply section 51 supplies a video to be displayed by thevideo display apparatus 40. In the present embodiment, we assume thatthe video supply section 51 supplies a planar video (2D video) M as avideo to be displayed. It should be noted that the video supply section51 may supply not only the planar video M but also a stereoscopicallyviewable stereoscopic video using parallax to the video displayapparatus 40.

Specifically, for example, the video supply section 51 may be realizedas a result of execution of an application program of a game by thecontrol section 11. In this case, the video supply section 51 supplies,as the planar video M, a game image indicating the state of progress ofthe game. The video supply section 51 may also be a rendering enginethat reads video data such as movies stored on an information storagemedium such as optical disc or hard disk and renders the video data.

Both of the first and second display control sections 52 and 53 realizedisplay control for causing the video display apparatus 40 to displaythe planar video M supplied by the video supply section 51 by outputtingthe planar video M to the relay apparatus 30. It should be noted,however, that the first and second display control sections 52 and 53cause the video display apparatus 40 to display the planar video M indifferent ways. In the description given below, the display moderealized by the first display control section 52 will be referred to asa first display mode, and the display mode realized by the seconddisplay control section 53 will be referred to as a second display mode.Also, a video including the planar video M and to be output to the relayapparatus 30 by each of the first and second display control sections 52and 53 will be referred to as a display video.

The first display mode is a display mode for displaying the planar videoM in the front direction of the user irrespective of the change inposition and/or orientation of the video display apparatus 40. In thefirst display mode, the planar video M is always displayed in thedisplay area of the video display apparatus 40 irrespective of thechange in position and orientation of the video display apparatus 40.Specifically, the first display control section 52 generates videos forright and left eyes in which the video supplied by the video supplysection 51 is placed at a given position in the approximate center.Then, the two generated videos are output to the relay apparatus 30 asdisplay videos. As a result, while the videos are displayed in the firstdisplay mode, the display position of the planar video M in the displayarea of the video display apparatus 40 remains unchanged so that theplanar video M is always displayed in the front direction as seen fromthe user. That is, while the videos are displayed in the first displaymode, the planar video M will never move outside the display area of thevideo display apparatus 40. Because the video display apparatus 40 issecured to the user's head, the user feels as if the screen displayingthe planar video M is moving in step with his or her own motion when theuser moves his or her own head to change the face position andorientation.

The second display mode is a display mode for displaying the planarvideo M in a manner that changes with the change in position and/ororientation of the video display apparatus 40. In the second displaymode, the display position of the planar video M in the display area ofthe video display apparatus 40 changes with movement of the videodisplay apparatus 40. Specifically, the second display control section53 controls the display position of the planar video M such that thedisplay position of the planar video M is fixed in place in a virtualspace. In the second display mode, therefore, when the usersignificantly moves his or her head, the planar video M may move outsidethe display area of the video display apparatus 40, possibly resultingin the planar video M not being displayed. In such a case, the seconddisplay control section 53 may include, for example, a marker imageindicating a direction in the display video to inform the user in whichdirection he or she should point his or her face so as to display theplanar video M again.

FIG. 5 is a diagram describing a specific example of a drawing processof the display video in the second display mode. This diagramillustrates a virtual space with a virtual screen VS and two viewpointcameras C1 and C2 placed therein as seen from above. As illustrated inFIG. 5 , the second display control section 53 has the virtual screen VSplaced in the virtual space and the two viewpoint cameras C1 and C2placed at a position opposed to the virtual screen VS. The two viewpointcameras C1 and C2 are horizontally arranged side by side and at a givendistance from each other. Then, the planar video M supplied by the videosupply section 51 is attached to the front face of the virtual screen VSas texture. In this state, the second display control section 53generates a video for left eye by drawing an image indicating how theinside of the virtual space looks as seen from the viewpoint camera C1.The second display control section 53 also generates a video for righteye by drawing an image indicating how the inside of the virtual spacelooks as seen from the viewpoint camera C2. As the video displayapparatus 40 displays display videos generated as described above, theuser can view the planar video M appearing on the virtual screen VS thatis located at a position in front of and at a distance from the user.

Further, the second display control section 53 changes the orientationof the two viewpoint cameras C1 and C2 (line-of-sight direction) withchange in orientation of the video display apparatus 40. Here, theorientation of the video display apparatus 40 may be defined, forexample, by rotational angles (e.g., yaw, pitch, and roll angles) aboutrotational axes that run along the vertical, horizontal, andlongitudinal directions of the video display apparatus 40, respectively.The second display control section 53 changes the position of the twoviewpoint cameras C1 and C2 in response to translation of the videodisplay apparatus 40. As described earlier, the second display controlsection 53 can identify the change in position and orientation of thevideo display apparatus 40 by using measurement results of the motionsensor 44 and detecting the position of light of the light-emittingdevice 43 from the image captured by the camera 25. The second displaycontrol section 53 changes the position and orientation of the viewpointcameras C1 and C2 with change in position and orientation of the videodisplay apparatus 40 identified in this manner.

Specifically, when the orientation of the video display apparatus 40changes, the second display control section 53 rotates the viewpointcameras C1 and C2 in the same rotational direction as that rotation andby the same amount of rotation. At this time, the viewpoint cameras C1and C2 rotate in the virtual space about the center position between thetwo as a rotational axis while maintaining a distance from each other.Also, when the video display apparatus 40 translates from a relativeposition, the second display control section 53 moves both the viewpointcameras C1 and C2 in the virtual space in response to the direction ofmotion and the travel thereof. Then, the second display control section53 also generates videos for left and right eyes by drawing how thevirtual screen VS looks as seen from the viewpoint cameras C1 and C2whose position and orientation have been changed. At this time, theposition and orientation of the viewpoint cameras change, but theposition of the virtual screen VS in the virtual space remainsunchanged. This allows the user to experience a sensation as if thescreen indicating the planar video M were actually fixed in the airwhile viewing the planar video M. Further, the user can view the planarvideo M while focusing attention on what appears in the area near theperimeter which is difficult to visually recognize in the initialcondition by changing the orientation and position of his or her facesuch that the area near the perimeter of the planar video M is locatedat the center of the field of view, for example.

It should be noted that although, here, the display videos are updatedwith change in both orientation and position of the video displayapparatus 40, the second display control section 53 may change thedisplay videos only with change in orientation or position of the videodisplay apparatus 40. Also, in the above description, planar displayvideos were generated by drawing how the virtual screen VS placed in avirtual 3D space looked as seen from the two viewpoint cameras. However,the display video drawing process is not limited to such a method, andthe second display control section 53 may generate display videos by 2Dimage processing. For example, the second display control section 53determines the position where the planar video M is placed in thedisplay video in response to the change in position of the video displayapparatus 40 and draws videos for left and right eyes, each with theplanar video M placed at the determined position. At this time, thesecond display control section 53 changes the position of the planarvideo M in the display video such that the video display apparatus 40moves in the opposite direction to the direction of movement of thevideo display apparatus 40. This allows the user to feel as if theplanar video M remained at the original position even if the user moveshis or her head. Further, the second display control section 53 maychange the display position of the planar video M in response to thechange in orientation of the video display apparatus 40 and change theshape of the planar video M, for example, through projectivetransformation. According to such control, it is possible to display theplanar video M in the second display mode without carrying out 3D-basedimage processing.

FIGS. 6A to 6C are diagrams describing the difference in display betweenthe first and second display modes. FIG. 6A illustrates an example ofdisplay in an initial state (state in which the display of the planarvideo M begins). In the initial state, the display is the similar in thefirst and second display modes. In contrast, FIGS. 6B and 6C illustratean example of display when the user tilts his or her head from the statedepicted in FIG. 6A. FIGS. 6B and 6C illustrate an example of display inthe first display mode and an example of display in the second displaymode, respectively. In the first display mode, even when the user tiltshis or her head, the display position of the planar video M in thedisplay area of the video display apparatus 40 remains unchanged. Forthis reason, when the user tilts his or her head, it looks as if theplanar video M were also tilted in response thereto. In contrast, in thesecond display mode, the display position of the planar video M in thevirtual space is controlled such that the position remains unchanged asdescribed earlier irrespective of the movement of the user's face.Therefore, when the user tilts his or her head, the display position ofthe planar video M in the display area of the video display apparatus 40rotates as illustrated in FIG. 6C. As a result, it looks as if theplanar video M maintained the same orientation as in FIG. 6A as seenfrom the user.

The switching control section 54 performs display switching control forswitching between the first and second display modes that have beendescribed up to this point based on a given condition. In the presentembodiment in particular, the switching control section 54 may switchthe display mode in response to a direct instruction input to theoperating device 20 from the user but may also switch the display modebased on various other conditions. A description will be given below ofseveral specific examples of conditions for the switching controlsection 54 to switch the display mode.

First, as a first example of a switching condition, an example ofswitching the display mode in accordance with the display size of theplanar video M will be described. In this example, the display size ofthe planar video M may be switched in accordance with an instructioninput to the operating device 20 from the user or may be switched by acontrol input from the video supply section 51 that supplies the planarvideo M. Also, the default display size may be specified in advance bythe user.

In this example, the switching control section 54 determines in whichdisplay mode, the first or second display mode, the planar video M is tobe displayed in response to the display size of the planar video M to bedisplayed from now at a time when the display of the planar video Mbegins or when the display size of the planar video M is changed. Then,the switching control section 54 instructs the first display controlsection 52 or the second display control section 53 to display theplanar video M in the determined display mode.

More specifically, the switching control section 54 selects the seconddisplay mode when the planar video M should be displayed in a displaysize equal to a given threshold or more. Conversely, the switchingcontrol section 54 selects the first display mode when the planar videoM should be displayed in a display size less than the given threshold. Alarge display size may result in the planar video M failing to fit intothe display area of the video display apparatus 40. Further, the planarvideo M equal to or smaller than the display area may lead to distortionin the displayed image at a position near the perimeter of the displayarea, resulting in difficulty for the user to visually recognize theimage near the perimeter. For this reason, when the display size isequal to a given threshold or more, the planar video M is displayed inthe second display mode. In the second display mode, the position andorientation of the viewpoint cameras change in accordance with themovement of the user's head. For this reason, the user can view theplanar video M in a manner that permits easy viewing of the area nearthe perimeter by moving the area near the perimeter of the planar videoM to somewhere near the center of the display area of the video displayapparatus 40. Conversely, when the display size of the planar video M issmall, the user can view the entire planar video M without moving theviewpoint. Therefore, the user can always view the entire planar video Mby displaying the planar video M in the first display mode fordisplaying the planar video M in the front direction of the user.

A description will be given next of display switching control performedwhen the user begins to use the video display apparatus 40 as a secondexample of a switching condition. In this second example, theinformation processing apparatus 10 detects the user's attempt to startusing the video display apparatus 40 based on detection results of theproximity sensor 45. Then, when the user starts using the video displayapparatus 40, the information processing apparatus 10 displays theplanar video M including a startup message or a system menu, forexample, on the video display apparatus 40.

In this example, the switching control section 54 instructs the firstdisplay control section 52 to display the planar video M in the firstdisplay mode at a time when the proximity sensor 45 detects the presenceof an object for the first time. Then, when a given time period elapsesfrom the detection of presence of an object by the proximity sensor 45for the first time, the switching control section 54 instructs thesecond display control section 53 to switch to the second display mode.Such switching control is performed for the following reason. That is,when the proximity sensor 45 detects the presence of an object for thefirst time, it is highly likely that the user has yet to wear the videodisplay apparatus 40 at its correct position and is still in the middleof putting on the video display apparatus 40. Therefore, it is notpreferred to finalize the display position of the planar video Mimmediately when the proximity sensor 45 detects the presence of anobject. On the other hand, if nothing is displayed while the user isattempting to put on the video display apparatus 40, it will becomedifficult to adjust the position of the video display apparatus 40 whenthe user puts on the video display apparatus 40. For this reason, in thesecond example, the planar video M is displayed in the first displaymode first when the user begins to put on the video display apparatus40. In the first display mode, the planar video M is always displayedapproximately at the center of the display area of the video displayapparatus 40. As a result, even if the user moves his or her head whileputting on the video display apparatus 40, the display position in thedisplay area of the planar video M remains unchanged. This allows theuser to adjust the position of the video display apparatus 40 such thatthe planar video M is located at an easy-to-see position. Thereafter,the display mode is switched to the second display mode at a time whenthe user is probably finished with putting on the video displayapparatus 40, making it possible to control the display position of theplanar video M relative to the position of the video display apparatus40 at that time.

A description will be given here of an example of a processing flowperformed by the information processing apparatus 10 in this secondexample with reference to the flowchart depicted in FIG. 7 . In thestate where the video display apparatus 40 is not used, the switchingcontrol section 54 acquires detection results of the proximity sensor 45on a regular basis (S1) and verifies whether the presence of an objecthas not been detected (S2). When the presence of an object is detectedin S2, it is probable that the user is beginning to use the informationprocessing apparatus 10. Therefore, the switching control section 54instructs the first display control section 52 to initiate the displayin the first display mode. As a result, the first display controlsection 52 initiates a process of displaying the planar video M such assystem menu in the first display mode (S3).

Thereafter, the switching control section 54 counts the elapsed timefrom the start of the planar video M display process in S3 anddetermines whether a given time period has elapsed (S4). When the giventime period has elapsed, the switching control section 54 instructs thesecond display control section 53 to initiate the display in the seconddisplay mode. As a result, the second display control section 53initiates a process of displaying the planar video M that has beendisplayed in the first display mode up to that time in the seconddisplay mode (S5).

At this time, the second display control section 53 generates a displayvideo by using, as a reference posture of the video display apparatus40, the position and orientation of the video display apparatus 40 atthe time when the display process in S5 was initiated. That is, theposition and orientation of the video display apparatus 40 at the timeof switching from the first display mode to the second display mode areassociated with the initial position and line-of-sight direction of theviewpoint cameras in the virtual space, and the virtual screen VS isplaced at the front in the line-of-sight direction of the viewpointcameras. As a result, when the second display mode is initiated, theplanar video M is displayed in the front direction of the user as in thefirst display mode. Thereafter, as the position and orientation of thevideo display apparatus 40 change, the position and line of sight of theviewpoint cameras will change in response to the change. As a result,assuming that the state after the elapse of a given time period fromwhen the user begins to wear the video display apparatus 40 (i.e., statein which the user is probably finished with putting on the video displayapparatus 40) is a reference state, it is possible to fix the virtualscreen VS in place in the front direction of the user in this referencestate.

Also, in the second example, the first display control section 52 mayinclude, in the display video, a notification video for informing theuser of time remaining until the first display mode is switched to thesecond display mode. The notification video in this case may be, forexample, a time counter directly indicating, in number form, elapsedtime from the start of the display in the first display mode and timeremaining until the display mode will be switched and a gauge imagevisually indicating elapsed time.

The first and second examples described above may be used incombination. Specifically, if the display size of the planar video M isequal to a given threshold or more, the switching control section 54displays the planar video M in the first display mode at a time when theproximity sensor 45 detects the presence of an object for the first timeand switches the display mode to the second display mode when a giventime period elapses as described in the second example. On the otherhand, if the display size of the planar video M is less than the giventhreshold, such switching is not performed, and the switching controlsection 54 continues to display the planar video M in the first displaymode.

It should be noted that the condition for triggering the switchingbetween the display modes by the switching control section 54 is notlimited to those described above. For example, the switching controlsection 54 may switch the display mode in response to an expressinstruction from the video supply section 51. The video supply section51 instructs the switching control section 54 to switch between thefirst and second display modes, for example, in accordance with thenature of content included in the planar video. This makes it possibleto display the planar video M in the display mode appropriate fordetails of content to be displayed.

The information processing apparatus 10 according to the presentembodiment described above makes it possible to display the planar videoM in the display mode appropriate for user's circumstances of use anddetails of content by switching between the first and second displaymodes in accordance with various conditions.

It should be noted that embodiments of the present invention are notlimited to that described above. For example, although the first displaycontrol section 52 generated videos for left and right eyes by itself inthe above description, the present invention is not limited thereto, andthe first display control section 52 may output the planar video Msupplied by the video supply section 51 to the relay apparatus 30 in an‘as-is’ manner as a display video. In this case, the relay apparatus 30or the video display apparatus 40 generates videos for left and righteyes having the supplied planar video M placed approximately at thecenter. According to such a method, the relay apparatus 30 or the videodisplay apparatus 40 can accept a video signal including the normalplanar video M supplied by a conventional information processingapparatus and display the signal on the video display device 41.

Also, the first display control section 52 may generate a display videofor the first display mode by using a process similar to generation of adisplay video for the second display mode by the second display controlsection 53. Specifically, the first display control section 52 may placethe virtual screen VS and the viewpoint cameras C1 and C2 in a virtualspace and generate a video indicating how the virtual screen VS looks asseen from each of the viewpoint cameras C1 as the display video. In thiscase, the first display control section 52 can maintain the position ofthe planar video M unchanged in the display video by keeping theposition of the viewpoint cameras C1 and C2 fixed in place at all timesirrespective of the change in position and orientation of the videodisplay apparatus 40. Such control makes it possible to supply a displayvideo to the relay apparatus 30 as a video signal in the similar formatgenerated by the similar procedure without switching the drawing processbetween the first and second display modes.

Also, part or whole of the processes performed by the informationprocessing apparatus 10 in the above description may be realized by therelay apparatus 30. In this case, the system as a whole including theinformation processing apparatus 10 and the relay apparatus 30 or therelay apparatus 30 alone functions as the information processingapparatus according to the present invention.

REFERENCE SIGNS LIST

-   -   1: Video display system, 10: Information processing apparatus,        11: Control section, 12: Storage section, 13: Interface section,        20: Operating device, 25: Camera, 30: Relay apparatus, 40: Video        display apparatus, 41: Video display device, 42: Optical device,        43: Light-emitting device, 44: Motion sensor, 45: Proximity        sensor, 46: Communication interface, 51: Video supply section,        52: First display control section, 53: Second display control        section, 54: Switching control section

The invention claimed is:
 1. An information processing apparatus adaptedto be connected to an image display apparatus worn on a head of a user,the information processing apparatus comprising: an image supplierconfigured to produce a planar image representing virtual objects in avirtual world, wherein the planar image includes characteristics thatrequire the image display apparatus to be a non-see-through imagedisplay apparatus that does not allow the user to visually recognizereal world objects outside the virtual world; and a display controllerconfigured to display the planar image on the image display apparatus ina display mode that changes a shape of the planar image as a function ofa change in position and/or orientation of the image display apparatus,wherein the change in the shape of the planar image includes presentingthe planar image in an orientation other than a front direction of theuser.
 2. The information processing apparatus of claim 1, wherein thedisplay controller changes the shape of the planar image throughprojective transformation.
 3. The information processing apparatus ofclaim 1, further comprising an operating device, including touch panel,that issues instructions to the information processing apparatus tochange a display size of the planar image.
 4. The information processingapparatus of claim 1, wherein the planar image is a system menu.
 5. Theinformation processing apparatus of claim 1, wherein the displaycontroller displays a marker image to the user indicating a changeposture of the user.
 6. The information processing apparatus of claim 5,wherein the marker image indicates a direction.
 7. The informationprocessing apparatus of claim 1, further comprising: another displaycontroller configured to display the planar image on the image displayapparatus in another display mode in which the planar image is displayedin a front direction of the user irrespective of the change in positionand/or orientation of the image display apparatus; and a switchingcontroller configured to switch between the display mode and the otherdisplay mode based on a given condition.
 8. A method of controlling aninformation processing apparatus that is connected to an image displayapparatus worn on a head of a user, the method comprising: producing aplanar image representing virtual objects in a virtual world, whereinthe planar image includes characteristics that require the image displayapparatus to be a non-see-through image display apparatus that does notallow the user to visually recognize real world objects outside thevirtual world; and displaying the planar image on the image displayapparatus in a display mode that changes a shape of the planar image asa function of a change in position and/or orientation of the imagedisplay apparatus, wherein the change in the shape of the planar imageincludes presenting the planar image in an orientation other than afront direction of the user.
 9. The method of claim 8, wherein thedisplaying includes changing the shape of the planar image throughprojective transformation.
 10. The method of claim 8, further comprisingissuing instructions from an operating device, including touch panel, tothe information processing apparatus to change a display size of theplanar image.
 11. The method of claim 8, wherein the planar image is asystem menu.
 12. The method of claim 8, wherein the displaying includesdisplaying a marker image to the user indicating a change posture of theuser.
 13. The method of claim 12, wherein the marker image indicates adirection.
 14. The method of claim 8, further comprising: displaying theplanar image on the image display apparatus in another display mode inwhich the planar image is displayed in a front direction of the userirrespective of the change in position and/or orientation of the imagedisplay apparatus; and switching between the display mode and the otherdisplay mode based on a given condition.
 15. A non-transitory, computerreadable storage medium containing a computer program, which whenexecuted by a computer, causes the computer to carry out actions forcontrolling an information processing apparatus that is connected to animage display apparatus worn on a head of a user, the actionscomprising: producing a planar image representing virtual objects in avirtual world, wherein the planar image includes characteristics thatrequire the image display apparatus to be a non-see-through imagedisplay apparatus that does not allow the user to visually recognizereal world objects outside the virtual world; and displaying the planarimage on the image display apparatus in a display mode that changes ashape of the planar image as a function of a change in position and/ororientation of the image display apparatus, wherein the change in theshape of the planar image includes presenting the planar image in anorientation other than a front direction of the user.